An Efficient and Balanced Platform for Data-Parallel Subsampling Workloads

....................................................................................................................................ii Dedication ................................................................................................................................ iv Acknowledgments .................................................................................................................... v Vita ........................................................................................................................................... vi List of Tables ........................................................................................................................... ix List of Figures........................................................................................................................... x Chapter 1: Introduction ........................................................................................................... 1 1.1 Challenges in processing subsampling workloads .................................................. 3 1.2 Platforms and Workloads ......................................................................................... 5 1.3 Contributions ............................................................................................................. 6 Chapter 2: Related Work ........................................................................................................ 8 Chapter 3: System Design .................................................................................................... 12 3.1 Map-Reduce for subsampling workloads .............................................................. 12 3.2 Tiny task model for subsampling workloads ........................................................ 16 3.3 Job-level Recovery.................................................................................................. 24 3.4 Platform Selection ................................................................................................... 26 viii 3.5 Scalable distributed file system .............................................................................. 30 Chapter 4: Experimental Setup and Results ........................................................................ 32 4.1 Experimental Setup ................................................................................................. 32 4.2 Experimental Results .............................................................................................. 37 Chapter 5: Conclusion............................................................................................................ 49 References............................................................................................................................... 51